Algorithmic Information Theory and Foundations of Probability

The question how and why mathematical probability theory can be applied to the “real world” has been debated for centuries. We try to survey the role of algorithmic information theory (Kolmogorov complexity) in this debate. 1 Probability theory paradox One often describes the natural sciences framework as follows: a hypothesis is used to predict something, and the prediction is then checked against the observed actual behavior of the system; if there is a contradiction, the hypothesis needs to be changed. Can we include probability theory in this framework? A statistical hypothesis (say, the assumption of a fair coin) should be then checked against the experimental data (results of coin tossing) and rejected if some discrepancy is found. However, there is an obvious problem: The fair coin assumption says that in a series of, say, 1000 coin tossings all the 21000 possible outcomes (all 21000 bit strings of length 1000) have the same probability 2−1000. How can we say that some of them contradict the assumption while other do not? The same paradox can be explained in a different way. Consider a casino that wants to outsource the task of card shuffling to a special factory that produced shrink-wrapped well shuffled decks of cards. This factory would need some quality control department. It looks at the deck before shipping it to the customer, blocks some “badly shuffled” decks and approves some others as “well shuffled”. But how is it possible if all n! orderings of n cards have the same probability? 2 Current best practice Whatever the philosophers say, statisticians have to perform their duties. Let us try to provide a description of their current “best practice” (see [7, 8]). A. How a statistical hypothesis is applied. First of all, we have to admit that probability theory makes no predictions but only gives recommendations: if the probability (computed on the basis of ∗LIF Marseille, CNRS & Univ. Aix–Marseille. On leave from IITP, RAS, Moscow. Supported in part by NAFIT ANR-08-EMER-008-01 grant. E-mail: [email protected]